Improving Decision Trees by Clustering

Multi-modal classification problems arise in many fields and form an important class of problems. The presence of disjoint areas for each class creates special problems for techniques that cannot partition each class into more than one region. Among the various techniques that have been applied with some success to multi-modal problems are decision tree classifiers (DTCs) and back propagation neural networks. DTCs use a recursive partitioning algorithm to partition the feature space into disjoint areas. In principle, DTCs should perform well on multi-modal data sets because they have the ability to partition each class into disjoint areas. In practice, however, DTCs can have significant difficulty with such data sets. In the presence of multimodality decision trees can became very bushy leading to high error rates. We show that clustering of the data before classification can lead to simpler trees and lower error rates.